Local receptive field constrained deep networks

Automatic extraction of distinctive features from a visual information stream is challenging due to the large amount of information contained in most image data. In recent years deep neural networks (DNNs) have gained outstanding popularity for solving visual information processing tasks. This study reports novel contributions, including a new DNN architecture and training method, which increase the fidelity of DNN-based representations to encodings extracted by visual processing neurons. Our local receptive field constrained DNNs (LRF-DNNs) are pre-trained with a modified restricted Boltzmann machine, the LRFRBM, which utilizes biologically inspired Gaussian receptive field constraints to encourage the emergence of local features. Moreover, we propose a method for concurrently finding advantageous receptive field centers, while training the LRF-RBM. By utilizing LRF-RBMs with gradually increasing receptive field sizes on each layer, our LRF-DNN learns features of increasing complexity and demonstrates hierarchical part-based compositionality. We show superior face completion and reconstruction results on the challenging LFW face dataset.